Computing technology has revolutionized the way that we work, play, and communicate. Computing systems consist of interacting hardware and software. As far as hardware, computing systems are increasing taking a wide variety of physical forms including desktop computers, laptop computers, Personal Digital Assistants (PDAs), mobile phones, and even many devices (such as refrigerators) that have not conventionally been integrated with computing technology. Computing systems may even be networked together to form a distributed computing system. Thus, a computing system may technically be distributed throughout the globe, or may be small enough to fit in the palm of one's hand. Regardless of its physical form, a computing system often includes one or more processors, memory, and persistent storage.
On the other hand, software is a collection of computer-executable instructions and associated data that may be executed and used by the processor(s) to direct the computing system to perform in particular ways. Currently, there is a wide variety of software available for a wide variety of computing platforms. Accordingly, general purpose computing systems in particular may perform a wide variety of functionality.
A common way of changing the functionality of a computing system is to impose a hardware or software change on the computing system. A hardware change may include, for example, the installation of more memory, a new hard drive, another processor, external devices, or the like. A software change may include, for example, the installation or upgrade of new software. For instance, the installation or upgrade of a software application, operating system, or other software component would cause a functional change in the computing system. The installation of patches and services packs, for example, is an example of a software change as well.
Regardless of the type of hardware or software change, it is often quite difficult if not impossible, to predict beforehand the full effect of imposing a change in a particular environment. This is particularly true if the change and/or the deployment environment is complex. Conventionally, Information Technology (IT) professionals typically deploy a software change, for example, on a test machine and observe the subsequent behavior of the machine to form an opinion about the change's stability and compatibility with their production environment. If the change seems to have no adverse effects, the software change may eventually rolled out to the actual computing system or computing systems that are to have the change. If problems occur after the rollout, the rollout may typically be reversed.
The assessment process prior to rolling out a software change may be quite extensive and require significant human intervention and judgment as to the performance of the software change. Furthermore, the assessment period delays the overall rollout of the software change, and may not ultimately be a correct assessment since instability may be caused by non-obvious factors that may differ from one computing system to another.
There are conventional test suites that automatically test a software change prior to rollout. However, these conventional test suites typically provide only a success or failure decision, with little more information regarding the performance and stability of the software change. Accordingly, what would be advantageous is a mechanism that allows the stability of a hardware or software change to be assessed in a more automated and detailed fashion.